DAGGER: A New Approach to Combining Multiple Models Learned from Disjoint Subsets

We introduce a new technique for combining multiple learned models. This technique results in a single comprehensible model. This is to be contrasted with current methods that typically combine models by voting. The core of the technique, the DAGGER (Disjoint Aggregation using Example Reduction) algorithm selects examples which provide evidence for each decision region within each local model. A single model is then learned from the union of these selected examples. We describe experiments on models learned from disjoint training sets that show: (i) DAGGER performs as well as weighted voting on this task; (ii) DAGGER extracts examples which are more informative than those that can be selected at random. The experiments were conducted on models learned from disjoint subsets generated with a uniform random distribution. DAGGER is actually designed for use on naturally distributed tasks, with non-random distribution. We discuss how one view of the experimental results suggests that DAGGER should work well on this type of problem.